DOE_Micromonas : Configuring nextgen to run sequenceserver and genome-browser

This page last changed on Jan 16, 2019 by mccann.

Initial Virtual Machine setup

Requested VM from with Help Ticket 18084:

Request_Submitted_By: mccann@mbari.org 
VM_Name: nextgen 
VM_Purpose: Run genomics processing and server software 
VM_Expiration: Permanent 
VM_Support: IS_Supported 
Alt_Administrator: 
Project_Number: 
VM_OS: CentOS 6 - Recommended for Linux 
RAM: 2 
Addl_RAM_Reason: 
CPU: 2 
Addl_CPU_Reason: 
VM_disk: Data (as needed) may be mounted from atlas.  The default 75 GB should be fine to start with. 
VM_Comments: We envision this server to eventually perform the function that the wlab-hm "cluster" currently 
performs, but in a more robust manner. Processors and RAM may need to be added as we monitor it. 
The system needs to be configured by 16 December. 
Submit: Submit

Followed up with Help Tickets 18119, 18120, 18124, 18125, 18127, 18143, 18219, 18222, 18223, 18253, and 18255
CNAMEs of sequenceserver.shore.mbari.org and genomebrowser.shore.mbari.org configured to point to nextgen's IP address
sudo privileges: mccann, rich, and brian; sudo /sbin/service granted to user geneadm

SequenceServer - a Ruby on Rails Web application

See http://www.sequenceserver.com/
Built Ruby 1.9.3p484 from source following instructions at http://techblogthing.blogspot.com/2012/11/complete-guide-to-install-ruby-193-on.html

Installed sequenceserver with:

sudo /usr/local/bin/gem install sequenceserver
sudo /usr/local/bin/gem install passenger
sudo /usr/local/bin/gem install rspec
sudo /usr/local/bin/gem install watir-webdriver
sudo /usr/local/bin/gem install rack-test
sudo /usr/local/bin/gem install headless
sudo /sbin/chkconfig httpd on
sudo /sbin/service httpd start

Followed the "Optional: SequenceServer on Apache or Nginx with Phusion Passenger" instructions from http://www.sequenceserver.com/ as user mccann:
```
/usr/local/lib/ruby/gems/1.9.1/gems/passenger-4.0.29/bin/passenger-install-apache2-module
```
and followed instrcitions, it required several more 'sudo yum install's

Created file /etc/httpd/conf.d/sequenceserver.conf:

LoadModule passenger_module /usr/local/lib/ruby/gems/1.9.1/gems/passenger-4.0.29/buildout/apache2/mod_passenger.so
PassengerRoot /usr/local/lib/ruby/gems/1.9.1/gems/passenger-4.0.29
PassengerDefaultRuby /usr/local/bin/ruby

# Sub URL
<VirtualHost *:80>
        ServerName sequenceserver.shore.mbari.org
        DocumentRoot /usr/local/apache/htdocs
        <Directory /usr/local/apache/htdocs>
                AllowOverride All
                Options FollowSymLinks Indexes MultiViews
                Order allow,deny
                Allow from all
        </Directory>
        RailsBaseURI /sequenceserver-dbs
        RailsBaseURI /sequenceserver-meta
        RailsBaseURI /dbs
        RailsBaseURI /meta
        RailsEnv development
</VirtualHost>

Copied sequenceserver gem to additional directories to support multiple databases and symlinks for short names on sequenceserver.shore.mbari.org:

cd /usr/local/lib/ruby/gems/1.9.1/gems
sudo chown mccann .
cp -r sequenceserver-0.8.6 sequenceserver-dbs
cp -r sequenceserver-0.8.6 sequenceserver-meta
ln -s sequenceserver-dbs dbs
ln -s sequenceserver-meta meta
sudo chown root .
sudo chown -R root sequenceserver-dbs sequenceserver-meta

Create symlinks in /usr/local/apache/htdocs for web pages to work:

cd /usr/local/apache/htdocs
ln -s /usr/local/lib/ruby/gems/1.9.1/gems/sequenceserver-dbs/public sequenceserver-dbs
ln -s /usr/local/lib/ruby/gems/1.9.1/gems/sequenceserver-dbs/public dbs
ln -s /usr/local/lib/ruby/gems/1.9.1/gems/sequenceserver-meta/public sequenceserver-meta
ln -s /usr/local/lib/ruby/gems/1.9.1/gems/sequenceserver-meta/public meta

Modified /usr/local/lib/ruby/gems/1.9.1/gems/sequenceserver-dbs/config.ru, adding SequenceServer::App.config_file = ...

require 'rubygems'
require 'bundler/setup'
require 'sequenceserver'

SequenceServer::App.config_file = '/home/geneadm/conf/sequenceserver-dbs.conf'
SequenceServer::App.init
run SequenceServer::App

Modified /usr/local/lib/ruby/gems/1.9.1/gems/sequenceserver-meta/config.ru, adding SequenceServer::App.config_file = ...
```
require 'rubygems'
require 'bundler/setup'
require 'sequenceserver'

SequenceServer::App.config_file = '/home/geneadm/conf/sequenceserver-meta.conf'
SequenceServer::App.init
run SequenceServer::App
```
See the FAQ on http://www.sequenceserver.com/. We found that the config.ru files need to be owned by root for the servers to work.
Restart httpd and see the sequenceservers at:

http://sequenceserver.shore.mbari.org/dbs
http://sequenceserver.shore.mbari.org/meta
UCSC genome-browser - a collection of C cgi-bin programs

As user mccann on nextgen

sudo yum install mysql
sudo yum install mysql-server
sudo yum install mysql-devel
sudo yum install png-devel
sudo /sbin/chkconfig mysql on
sudo /sbin/service mysql start

Set the mysql root password

/usr/bin/mysql -u root -p 
update mysql.user set Password=PASSWORD('xxxx') where User='root'; 
# See Mike or Rich for the root password

As user geneadm on nextgen

mkdir ~/dev
cd ~/dev
git clone git://genome-source.cse.ucsc.edu/kent.git
cd kent

Follow instructions in the README and src/product/README.building.source, briefly:

export MACHTYPE=x86_64
mkdir -p ~/bin/${MACHTYPE}
cd src
make libs
cd gfServer
make
cd ../gfClient
make
cd ../blat
make
cd ../utils/faToNib
make
cd ../../hg
make
cd htdocs
make install
# Creates web site files in /usr/local/apache {cgi-bin-mccann, htdocs}

Setup apache server by creating a /etc/httpd/conf.d/genome-browser.conf file:

<VirtualHost *:80>
        ScriptAlias /cgi-bin/ "/usr/local/apache/cgi-bin-mccann/"
        ServerName genomebrowser.shore.mbari.org
        DocumentRoot "/usr/local/apache/htdocs/"
        <Directory "/usr/local/apache/htdocs">
        Options Includes Indexes FollowSymLinks
        AllowOverride None
        Order allow,deny
        Allow from all
                XBitHack on
        </Directory>

        <Directory "/usr/local/apache/cgi-bin-mccann">
        AllowOverride None
        Options None
        Order allow,deny
        Allow from all
        </Directory>

        <Directory "/usr/local/apache/htdocs/trash">
        Options MultiViews Indexes
        AllowOverride None
        Order allow,deny
        Allow from all
        </Directory>
</VirtualHost>

Restart httpd:
```
sudo /sbin/service httpd restart
```
and see the server at:

http://genomebrowser.shore.mbari.org/

Initial Virtual Machine setup

SequenceServer - a Ruby on Rails Web application

UCSC genome-browser - a collection of C cgi-bin programs